So we have our ltds base class and implementations

The current things i wish to address are that:

  • I have introduced versioning When I saved I was just writing the ltsd data into the data rep on Now if it is versioned I must construct the diffs here I must then do different saving actions depending on if it is new or not, I think this is a good task for the saving func because we can just do lil check for prescence

    If it is still unversioned then I must just save the data

  • Solve the whole noRefsInSerialisedData thing, including in the serialise funcs Can be solved in conjunction with instance specific data should the config store be deemed to be instance specific

  • Possibly unifying ram management or making it more modular Could call different So if designing for possibility of no memory management, would that still function? How would So it could be moved to the baseclass which could then call out to private functions which are implemented in the subclasses notfications seem ok, a lock combined with notifs probs to much to plan around emission could be before the action so listeners can prepare just beforehand, or maybe they want it after for new data idk could call the signal and then callers must call the corrosponding function during

    On writing a _writeFirstVersion function: We cant rely on ltsd init wanting to save and so we cant save during dataInit Unless we restrict non saving to only occur within the store and use a locking mechanism Although i maybe dont want it to be possible to write a store which doesnt save Well we are writing the dataInit func ourselves so we can d our initial save there Well we would still need to call out to implemented code in order to do the saving We could either call the base _saveData function and have that always handle initial states or we could create a _saveInitial function which takes a versioned: bool argument One thing that can also be done on initial save is to check for existing data with the desired id I prefer the _saveInitial method as it decreases the required work on the more often called _saveData function

    so _saveInitial is called from the _dataInit func, do we do version detection there

  • Cleanup terminology ltsd wrapped representation ( data that includes tags, id, ...) -> lTSDData data that is of interest representation -> data

    stored in quick memory -> {}QuickMem stored in long term memory -> {}LongStore

  • Logging transactions As data items, how? So I dont know if as data items but if so then the filter system can be used fairly easy if need but dont see the need atm

  • Many actions for saving, deletion, A lot of these would still require slow actions per item such as saving especially version diff calculations.

So to list current baseclass data and ( actions it implements and actions it doesnt), ignoring the extension protocols

Attribs

  • name

_initialSaveLock

Implemented actions

  • getCopy ->
  • getNonCentralCopy ->
  • getStoreIdPair ->
  • getDatasIntrinsicValue ->
  • _dataInit -> loads it into tracked mem, does initial ( blockable) save
  • _saveData -> handles versioned vs not, calls _writeNewVersion or _write
  • _deleteData ->
  • _reloadData ( might be able to make this generic using getLTSDFromStore)
  • getLTSD -> gets ltsd, calls implemented _getLTSDFromStore
  • getDatasAttributeValue -> finds it in mem, if not calls _getDatasAttributeValueFromStore
  • getQuickMemoryPrescence
  • _getDataFromQuickMemory

Non implemented actions

  • dataInit -> -
  • saveData -> -
  • deleteData -> -
  • reloadData -> -
  • doesDataExistInStore ->
  • getAllDataIdsInStore ->
  • getDatasTags ->
  • _getDatasNonStoreIntrinsicValue -> goes generic if we have generic mm, no bc implementation could still offer optimisation
  • getDatasAttributeValue -> -
  • getPythonObject -> -
  • getIdsWithQueryData ->
  • getQueryEditingUIAndInitialData ->
  • getVersionData -> if no id given, return latest, if specified, return all Do i need this? is it not created due to the store needing it to update? Upon saving we need to construct delta information, that means pulling in the data from the previous version. Now does this require a function to be implemented by the store? well if we want just the data then yes. If we can get by on searching for the previous commit using getDatasVersions and then using that to load in the ltsd, i think this will mean loading it as not tracked by store or finishing design on memory collection
  • _writeInitial ->
  • _writeInitialVersion ->
  • _write ->
  • _writeNewVersion -> should still update the tags
  • _deleteDataFromStore
  • getDatasVersions -> should return none for unversioned, or all
  • isDataVersioned -> could use getDatasVersions but this could be a significantly slower operation
  • getLTSDFromStore

Description of current ram management of data

So currently upon ltsd instanciation, it is saved to the instance dict If we request a load then if it is in the dict we pull from there if we delete then its deleted It is added when first pulling from the store

ref count dropping from store so when all the refs drop out of scope we remove it from

Communication between different instances

currently signals on save and delete

e.g. many places use the ltsii I may want to come in and update the list halfway through well

big concern here is data state new pickers up what to do in different scenarios